A Reactive Scheduling Strategy Applied On MapReduce OLAM Operators System
نویسندگان
چکیده
The combination of Data warehousing and data analysis techniques such as OLAP (Online Analytic Processing) and data mining through the Hadoop framework is an innovative way to treat large volumes of data. However, this way poses serious scheduling and combining tasks issues that bring more challenges. In this paper, we propose strategies to answer these questions, namely parallel OLAM (Online Analytic Mining) MapReduce Operators and a Reactive Scheduling Policy. OLAM MapReduce Operators divide jobs into two parts, the first includes all the operators that are used to create an OLAM CUBE and the second includes those who exploit the cube by data mining algorithms. The proposed policy coordinates the workflow generated by these operators, relying on model-based events. Our simulation experience shows that our strategy has a cumulative force that it reduces the execution time of the entire cluster at each request.
منابع مشابه
A Throughput Driven Task Scheduler for Batch Jobs in Shared MapReduce Environments
MapReduce is one of the most popular parallel data processing systems, and it has been widely used in many fields. As one of the most important techniques in MapReduce, task scheduling strategy is directly related to the system performance. However, in multi-user shared MapReduce environments, the existing task scheduling algorithms cannot provide high system throughput when processing batch jo...
متن کاملBased on the MapReduce Model for Data - intensive Computing of Energy Scheduling Algorithm Strategy
In this study, based on the consideration of energy consumption, we take to improve the strategy of the MapReduce job scheduling algorithm, in order to reduce the average response time for task scheduling of interactive jobs in the network. In accordance with the job priority grouping to adjust the scheduling task response time which can reduce the impact of network congestion, with good result...
متن کاملModified Pareto archived evolution strategy for the multi-skill project scheduling problem with generalized precedence relations
In this research, we study the multi-skill resource-constrained project scheduling problem, where there are generalized precedence relations between project activities. Workforces are able to perform one or several skills, and their efficiency improves by repeating their skills. For this problem, a mathematical formulation has been proposed that aims to optimize project completion time, reworki...
متن کاملAn evaluation of the performance of parallel database operators using Phoenix MapReduce
The database join operator is the most expensive operator of the relational algebra operators. Many highly efficient sequential and parallel operators exist, based on several core techniques: sort-merge, hash and nested-loops. We present the design and implementation of two parallel operators: an equi-join and a grouping aggregation. They utilise the emerging MapReduce paradigm, specifically a ...
متن کاملAn adaptive modified firefly algorithm to unit commitment problem for large-scale power systems
Unit commitment (UC) problem tries to schedule output power of generation units to meet the system demand for the next several hours at minimum cost. UC adds a time dimension to the economic dispatch problem with the additional choice of turning generators to be on or off. In this paper, in order to improve both the exploitation and exploration abilities of the firefly algorithm (FA), a new mo...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- JSW
دوره 7 شماره
صفحات -
تاریخ انتشار 2012